An EM-based Multi-Step Piecewise Surface Regression Learning Algorithm

A multi-step Expectation Maximization based (EM-based) algorithm is proposed to solve piecewise surface regression problem which has typical applications in market segmentation research, identification of consumer behavior patterns, weather patterns in meteorological research, and so on. The multiple steps involved are local regression on each data point of the training data set and a small set of its closest neighbors, clustering on the feature vector space formed from the local regression, regression learning for each individual surface, and classification to determine the boundaries for each individual surface. An EM-based iteration process is introduced in the regression learning phase to improve the learning outcome. The reassignment of cluster identifier for every data point in the training set is determined by predictive performance of each submodel. Cross validation technique is applied to the scenario in which the number of piecewise surfaces is not given in advance. A few clustering quality validity indexes such as Silhouette index and Davis-Bouldin index are adopted to estimate the number of piecewise surfaces as well. A set of experiments based on both artificial generated and benchmarks data source are conducted to compare the proposed algorithm and a few widely-used regression learning packages to show that the proposed algorithm outperforms those packages in terms of root mean squared errors (RMSE) of test data set.